Smart home appliances, operating method of thereof, and voice recognition system using the smart home appliances

ABSTRACT

Provided is a smart home appliance. The smart home appliance includes: a voice input unit collecting a voice; a voice recognition unit recognizing a text corresponding to the voice collected through the voice input unit; a capturing unit collecting an image for detecting a user&#39;s visage or face; a memory unit mapping the text recognized by the voice recognition unit and a setting function and storing the mapped information; and a control unit determining whether to perform a voice recognition service on the basis of at least one information of image information collected by the capturing unit and voice information collected by the voice input unit.

This application is a National Stage Application of InternationalApplication No. PCT/KR2014/010536, filed on Nov. 4, 2014, which claimsthe benefit of Korean Patent Application No. 10-2013-0153713, filed onDec. 11, 2013, all of which are hereby incorporated by reference intheir entirety for all purposes as if fully set forth herein.

TECHNICAL FIELD

The present disclosure relates to smart home appliances, an operatingmethod thereof, and a voice recognition system using the smart homeappliances.

BACKGROUND ART

Home appliances, as electronic products equipped in homes, includerefrigerators, air conditioners, cookers, and vacuum cleaner.Conventionally, in order to operate such home appliances, a method ofapproaching and directly manipulating them or remotely controlling themthrough a remote controller is used.

However, with the recent developments of communication technology, atechnique for inputting a command for operating home appliances by usinga voice and allowing the home appliances to recognize the inputted voicecontent and operate is introduced.

FIG. 1 is a view illustrating a configuration of a conventional homeappliance and its operating method.

A conventional home appliance includes a voice recognition unit 2, acontrol unit 3, a memory 4, and a driving unit 5.

When a user makes a voice meaning a specific command, the home appliance1 collects the spoken voice and interprets the collected voice by usingthe voice recognition unit 2.

As an interpretation result of the collected voice, a text correspondingto the voice may be extracted. The control unit 3 compares extractedfirst text information and second text information stored in the memory4 to determine whether the text is matched.

When the first and second text information matches, the control unit 3may recognize a predetermined function of the home appliance 1corresponding to the second text information.

Then, the control unit 3 may operate the driving unit 5 on the basis ofthe recognized function.

However, when such a conventional home appliance is in use, a noisesource generated from the surrounding may be wrongly recognized as avoice. Additionally, even when a user simply talks with other peoplenear a home appliance without an intention of speaking a command forvoice recognition, this may be also wrongly recognized. That is, thehome appliance malfunctions.

DISCLOSURE OF INVENTION Technical Problem

Embodiments provide a smart home appliance with improved voicerecognition rate, an operation method thereof, and a voice recognitionsystem using the smart home appliance.

Solution to Problem

In one embodiment, a smart home appliance includes: a voice input unitcollecting a voice; a voice recognition unit recognizing a textcorresponding to the voice collected through the voice input unit; acapturing unit collecting an image for detecting a user's visage orface; a memory unit mapping the text recognized by the voice recognitionunit and a setting function and storing the mapped information; and acontrol unit determining whether to perform a voice recognition serviceon the basis of at least one information of image information collectedby the capturing unit and voice information collected by the voice inputunit.

The control unit may include a face detection unit recognizing that auser is in a staring state for voice input when image information on auser's visage or face is collected for more than a setting time throughthe capturing unit.

The control unit may determine that a voice recognition service standbystate is entered when it is recognized that there is keyword informationin a voice through the voice input unit and a user in the staring statethrough the face detection unit.

The smart home appliance may further include: a filter unit removing anoise sound from the voice inputted through the voice input unit; and amemory unit mapping voice information related to an operation of thesmart home appliance and voice information unrelated to an operation ofthe smart home appliance in advance in the voice inputted through thevoice input unit and storing the mapped information.

The smart home appliance may further include: a region recognition unitdetermining a user's region on the basis of information on the voicecollected through the voice input unit; and an output unit outputtingregion customized information on the basis of information on a regiondetermined by the region recognition unit and information on the settingfunction.

The setting function may include a plurality of functions dividedaccording to regions; and the region customized information includingone function matching information on the region among the plurality offunctions is outputted through the output unit.

The output unit may output the region customized information by using adialect in the region determined by the region recognition unit.

The output unit may output a key word for security setting and the voiceinput unit may set a reply word corresponding to the key word.

The smart home appliance may further include an emotion recognition unitand an output unit, wherein the voice recognition unit may recognize atext corresponding to first voice information in the voice collectedthrough the voice input unit; the emotion recognition unit may extract auser's emotion on the basis of second voice information in the voicecollected through the voice input unit; and the output unit may outputuser customized information on information on a user's emotiondetermined by the emotion recognition unit and information on thesetting function.

The first voice information may include a language element in thecollected voice; and the second voice information may include anon-language element related to a user's emotion.

The emotion recognition unit may include a database where information onuser's voice characteristics and information on an emotion state aremapped; and the information on the user's voice characteristics mayinclude information on a speech spectrum having characteristics for eachuser's emotion.

The setting function may include a plurality of functions to berecommended or selected; and the user customized information includingone function matching the information on the user's emotion among theplurality of functions is outputted through the output unit.

The smart home appliance may further include: a position informationrecognition unit recognizing position information; and an output unitoutputting the information on the setting function on the basis ofposition information recognized by the position information recognitionunit.

The position information recognition unit may include: a GPS receptionunit receiving a position coordinate from an external positioninformation transmission unit; and a first communication modulecommunicably connected to a second communication module equipped in anexternal server.

The output unit may include a voice output unit outputting theinformation on the setting function as a voice by the positionrecognized by the position information recognition unit or a dialectused in a region.

The output unit may output information optimized for a region recognizedby the position information recognition unit among a plurality ofinformation on the setting function.

The position information recognized by the position informationrecognition unit may include weather information.

In another embodiment, an operating method of a smart home applianceincludes: collecting a voice through a voice input unit; recognizingwhether keyword information is included in the collected voice;collecting image information on a user's visage or face through acapturing unit equipped in the smart home appliance; and entering astandby state of a voice recognition service on the basis of the imageinformation on the user's visage or face.

When the image information on the user's visage or face is collected formore than a setting time, it may be recognized that a user is in astaring state for voice input; and when it is recognized that there iskeyword information in the voice and the user is in the staring statefor voice input, a standby state of the voice recognition service may beentered.

The method may further include: determining a user's region on the basisof information on the collected voice; and driving the smart homeappliance on the basis of information on the setting function andinformation on the determined region.

The method may further include outputting region customized informationrelated to the driving of the smart home appliance on the basis of theinformation on the determined region.

The outputting of the region customized information may includeoutputting a voice or a screen by using a dialect used in the user'sregion.

The method may further include performing a security setting, whereinthe performing of the security setting may include: outputting a set keyword; and inputting a reply word in response to the suggested key word.

The method may further include: extracting a user's emotion state on thebasis of information on the collected voice; and recommending anoperation mode on the basis of information on the user's emotion state.

The method may further include: recognizing an installation position ofthe smart home appliance through a position information recognitionunit; and driving the smart home appliance on the basis of informationon the installation position.

The recognizing of the installation position of the smart home appliancemay include receiving GPS coordinate information from a GPS satellite ora communication base station.

The recognizing of the installation position of the smart home appliancemay include checking a communication address as a first communicationmodule equipped in the smart home appliance is connected to a secondcommunication module equipped in a server.

In further another embodiment, a voice recognition system includes: amobile device including a voice input unit receiving a voice; a smarthome appliance operating and controlled based on a voice collectedthrough the voice input unit; and a communication module equipped ineach of the mobile device and the smart home appliance, wherein themobile device includes a movement detection unit determining whether toenter a standby state of a voice recognition service in the smart homeappliance by detecting a movement of the mobile device.

The movement detection unit may include an acceleration sensor or a gyrosensor detecting a change in an inclined angle of the mobile device,wherein the voice input unit may be disposed at a lower part of themobile device; and when a user puts the voice input unit close the mouthin order for a voice input as gripping the mobile device, an angle valuedetected by the acceleration sensor or the gyro sensor may be reduced.

The movement detection unit may include an illumination sensor detectingan intensity of an external light collected by the mobile device; andwhen a user puts the voice input unit close the mouth in order for avoice input as gripping the mobile device, an intensity value of a lightdetected by the illumination sensor may be increased.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features will be apparent fromthe description and drawings, and from the claims.

Advantageous Effects of Invention

According to the present invention, since a user controls an operationof a smart home appliance through a voice, usability may be improved.

Additionally, a user's face or an action for manipulating a mobiledevice is recognized and through this, whether a user has an intentionfor speaking a voice command is determined, so that voice misrecognitionmay be prevented.

Furthermore, when there are a plurality of voice recognition availablesmart home appliances, a command subject may be recognized by extractingthe feature (specific word) of a command voice that a user makes andonly a specific electronic product among a plurality of electronicproducts responds according to the recognized command subject.Therefore, miscommunication may be prevented during an operation of anelectronic product.

Additionally, whether a user speaks in standard language or direct isrecognized and according to the content or dialect type of therecognized voice, customized information is provided, so that user'sconvenience may be improved.

Moreover, since a setting on whether to use a voice recognition functionof a home appliance and a security setting are possible, an arbitraryuser may be prevented from using a corresponding function. Therefore,the reliability of a product operation may be increased.

Especially, happy, angry, sad, and trembling emotions may be classifiedfrom a user's voice by using mapped information of voice characteristicsand emotional states in voice and on the basis of the classifiedemotion, an operation of a home appliance may be performed orrecommended.

Additionally, since mapping information for distinguishing voiceinformation related to an operation of an air conditioner from voiceinformation related to noise in a user's voice is stored in the airconditioner, the misrecognition of a user's voice may be reduced.

According to the present invention, since a user controls an operationof a smart home appliance through a voice, usability may be improved.Thus, industrial applicability is remarkable.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view illustrating a configuration of a conventional homeappliance and its operating method.

FIG. 2 is a view illustrating a configuration of an air conditioner asone example of a smart appliance according to a first embodiment of thepresent invention.

FIGS. 3 and 4 are block diagrams illustrating a configuration of the airconditioner.

FIG. 5 is a flowchart illustrating a control method of a smart homeappliance according to a first embodiment of the present invention.

FIG. 6 is a schematic view illustrating a configuration of a voicerecognition system according to a second embodiment of the presentinvention.

FIG. 7 is a block diagram illustrating a configuration of a voicerecognition system according to a second embodiment of the presentinvention.

FIG. 8 is a flowchart illustrating a control method of a smart homeappliance according to a first embodiment of the present invention.

FIG. 9 is a view when a user performs an action for starting a voicerecognition by using a mobile device according to a second embodiment ofthe present invention.

FIG. 10 is a view illustrating a configuration of a plurality of smarthome appliances according to a third embodiment of the presentinvention.

FIG. 11 is a view when a user makes a voice on a plurality of smart homeappliances according to a third embodiment of the present invention.

FIG. 12 is a view when a plurality of smart home appliances operate byusing a mobile device according to a fourth embodiment of the presentinvention.

FIG. 13 is a view illustrating a configuration of a smart home applianceor a mobile device and an operating method thereof according to anembodiment of the present invention.

FIG. 14 is a view illustrating a message output of a display unitaccording to an embodiment of the present invention.

FIG. 15 is a view illustrating a message output of a display unitaccording to another embodiment of the present invention.

FIGS. 16A and 16B are views illustrating a security setting for voicerecognition function performance according to a fifth embodiment of thepresent invention.

FIG. 17 is a view illustrating a configuration of a voice recognitionsystem and its operation method according to a sixth embodiment of thepresent invention.

FIGS. 18 to 20 are views illustrating a message output of a display unitaccording to a sixth embodiment of the present invention.

FIGS. 21A to 23 are views illustrating a message output of a displayunit according to another embodiment of the present invention.

FIG. 24 is a block diagram illustrating a configuration of an airconditioner as one example of a smart home appliance according to aseventh embodiment of the present invention.

FIG. 25 is a flowchart illustrating a control method of a smart homeappliance according to a seventh embodiment of the present invention.

FIG. 26 is a view illustrating a display unit of a smart home appliance.

FIG. 27 is a view illustrating a configuration of a cooker as anotherexample of a smart home appliance according to a seventh embodiment ofthe present invention.

FIGS. 28A and 28B are views illustrating a display unit of the cooker.

FIG. 29 is a view illustrating a configuration of a washing machine asanother example of a smart home appliance according to an eightembodiment of the present invention.

FIG. 30 is a flowchart illustrating a control method of a smart homeappliance according to an eighth embodiment of the present invention.

FIG. 31 is a block diagram illustrating a configuration of a voicerecognition system according to a ninth embodiment of the presentinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, specific embodiments of the present invention are describedwith reference to the accompanying drawings. However, the idea of thepresent invention is not limited to suggested embodiments and thoseskilled in the art may suggest other embodiments within the scope of thesame idea.

FIG. 2 is a view illustrating a configuration of an air conditioner asone example of a smart appliance according to a first embodiment of thepresent invention. FIGS. 3 and 4 are block diagrams illustrating aconfiguration of the air conditioner.

Hereinafter, although an air conditioner is described as one example ofa smart appliance, it should be clear in advance that the ideas relatedto a voice recognition or communication (information offer) procedureexcept for the unique setting functions of an air conditioner may beapplied to other smart home appliances, for example, cleaners, cookers,washing machines, or refrigerators.

Referring to FIGS. 2 to 4, an air conditioner 10 according to the firstembodiment of the present invention includes a suction part 22,discharge parts 25 and 42, and a case 20 forming an external appearance.The air conditioner 10 shown in FIG. 2 may be an indoor unit installedin an indoor space to discharge air.

The suction part 22 may be formed at the rear of the case 20. Then, thedischarge parts 25 and 42 include a main discharge part 25 through whichthe air suctioned through the suction part 22 is discharged to the frontor side of the case 20 and a lower discharge part 42 discharging the airdownwardly.

The main discharge parts 25 may be formed at the both sides of the case20 and its opening/closing degree may be adjusted by a discharge vane26. The discharge vane 26 may be rotatably provided at one side of themain discharge part 25. The opening/closing degree of the lowerdischarge part 42 may be adjusted by a lower discharge vane 44.

A vertically movable upper discharge device 30 may be provided at anupper part of the case 20. When the air conditioner 10 is turned on, theupper discharge device 30 may move to protrude from an upper end of thecase 20 toward an upper direction and when the air conditioner 10 isturned off, the upper discharge device 30 may move downwardly and may bereceived inside the case 20.

An upper discharge part 32 discharging air is defined at the front ofthe upper discharge device 30 and an upper discharge vane 34 adjusting aflow direction of the discharged air is equipped inside the upperdischarge device 30. The upper discharge vane 34 may be providedrotatably.

A voice input unit 110 receiving a user's voice is equipped on at leastone side of the case 20. For example, the voice input unit 110 may beequipped at the left side or right side of the case 20. The voice inputunit 110 may be referred to as a “voice collection unit” in that it ispossible to collect voice. The voice input unit 110 may include amicrophone. The voice input unit 110 may be disposed at the rear of themain discharge unit 25 so as not to be affected by the air dischargedfrom the main discharge part 25.

The air conditioner 10 may further include a body detection unit 36detecting a body in an indoor space or a body's movement. For example,the body detection unit 36 may include at least one of an infraredsensor and a camera. The body detection unit 36 may be disposed at thefront part of the upper discharge device 30.

The air conditioner 10 may further include a capsule injection device 60through which a capsule with aroma is injected. The capsule input device60 may be installed at the front of the air conditioner 10 to bewithdrawable. When a capsule is inputted to the capsule input device 60,a capsule release device (not shown) disposed inside the air conditioner10 pops the capsule and a predetermine aroma fragrance is diffused.Then, the diffused aroma fragrance may be discharged to the outside ofthe air conditioner 10 together with the air discharged from thedischarge parts 25 and 42. According to types of the capsule, aromafragrance may be provided variously, for example, lavender, rosemary orpeppermint.

The air conditioner 10 may include a filter unit 115 for removing noisesound from a voice inputted through the voice input unit 110. The voicemay be filtered into a voice frequency easy for voice informationthrough the filter unit 115.

The air conditioner 10 includes control units 120 and 150 recognizinginformation for an operation of the air conditioner 10 from the voiceinformation passing through the filter unit 115. The control units 120and 150 include a main control unit 120 controlling an operation of thedriving unit 140 in order for an operation of the air conditioner 10 andan output unit 160 communicably connected to the main control unit 120and controlling the output unit 160 to display operation information ofthe air conditioner 10 to the outside.

The driving unit 140 may include a compressor or a blow fan. Then, theoutput unit 160 includes a display unit displaying operation informationof the air conditioner 10 as an image and a voice output unit outputtingthe operation information as a voice. The voice output unit may includea speaker. The voice output unit may be disposed at one side of the case20 and may be provided separated from the voice input unit 110.

Then, the air conditioner 10 may include a memory unit 130 mapping andstoring voice information related to an operation of the air conditioner10 and voice information irrelevant to an operation of the airconditioner 10 in advance among voices inputted through the voice inputunit 110. First voice information, second voice information, and thirdvoice information are stored in the memory unit 130. In more detail,frequency information defining the first to third voice information maybe stored in the memory unit 130.

The first voice information may be understood as voice informationrelated to an operation of the air conditioner 10, that is, keywordinformation. When it is recognized that the first voice information isinputted, an operation of the air conditioner 10 corresponding to theinputted first voice information may be performed or stop. The memoryunit 130 may include text information corresponding to the first voiceinformation and information on a setting function corresponding to thetext information.

For example, when the first voice information corresponds to ON of theair conditioner 10, as the first voice information is recognized, anoperation of the air conditioner 10 starts. On the other hand, when thefirst voice information corresponds to OFF of the air conditioner 10, asthe first voice information is recognized, an operation of the airconditioner 10 stops.

As another example, when the first voice information corresponds to oneoperation mode of the air conditioner 10, that is, air conditioning,heating, ventilation or dehumidification, as the first voice informationis recognized, a corresponding operation mode may be performed. As aresult, the first voice information and an operation method (on/off andan operation mode) of the air conditioner 10 corresponding to the firstvoice information may be mapped in advance in the memory unit 130.

The second voice information may include frequency information similarto the first voice information related to an operation of the airconditioner 10 but is substantially understood as voice informationrelated to an operation of the air conditioner 10. Herein, the frequencyinformation similar to the first voice information may be understood asfrequency information showing a frequency difference between the firstvoice information and the inside of a setting range.

When it is recognized that the second voice information is inputted,corresponding voice information may be filtered as noise information.That is, the main control unit 120 may recognize the second voiceinformation but may not perform an operation of the air conditioner 10.

The third voice information may be understood as voice informationirrelevant to an operation of the air conditioner 10. The third voiceinformation may be understood as frequency information showing afrequency difference between the first voice information and the outsideof a setting range. The second voice information and the third voiceinformation may be referred to as “unrelated information” in that theyare voice information unrelated to an operation of the air conditioner10.

In such a way, since voice information related to an operation of a homeappliance and voice information unrelated to an operation of a homeappliance are mapped and stored in advance in a smart home appliance,voice recognition may be performed effectively.

The air conditioner 10 includes a camera 50 as a capturing unitcapturing a user's face. For example, the camera 50 may be installed atthe front part of the air conditioner 10.

The air conditioner 10 may further include a face detection unit 180recognizing that a user looks at the air conditioner 10 in order forvoice recognition on the basis of an image captured through the camera50. The face detection unit 180 may be installed inside the camera 50 orinstalled separately in the case 20, in order for one function of thecamera 50. When the camera 50 captures a user's face for a predeterminedtime, the face detection unit 180 recognizes that a user stares at theair conditioner 10 in order for voice input (that is, a staring state).

The air conditioner 10 further include a voice recognition unit 170extracting text information from a voice collected through the voiceinput unit 110 and recognizing a setting function of the air conditioner10 on the basis of the extracted text information.

The information recognized by the voice recognition unit 170 or the facedetection unit 180 may be delivered to the main control unit 120. Themain control unit 120 may realize a user's intention of using a voicerecognition service and may then enter a standby state on the basis ofthe information recognized by the voice recognition unit 170 and theface detection unit 180.

Although the voice recognition unit 170, the face detection unit 180,and the main control unit 120 are separately configured as shown in FIG.4, the voice recognition unit 170 and the face detection unit 180 may beinstalled as one component of the main control unit 120. That is, thevoice recognition unit 170 may be understood as a function component ofthe main control unit 120 to perform a voice recognition function andalso the face detection unit 170 may be understood as a functioncomponent of the main control unit 120 to perform a face detectionfunction.

FIG. 5 is a flowchart illustrating a control method of a smart homeappliance according to a first embodiment of the present invention.

Referring to FIG. 5, in controlling a smart home appliance according tothe first embodiment of the present invention, a voice recognitionservice may be set to be turned on. The voice recognition service isunderstood as a service controlling an operation of the air conditioner10 by inputting a voice command. For example, in order to turn on thevoice recognition service, a predetermined input unit (not shown) may bemanipulated. Of course, when a user does not want to use a voicerecognition service, it may be set to be turned off in operation S11.

Then, a user speaks a predetermined voice command in operation S12. Thespoken voice is collected through the voice input unit 110. It isdetermined whether keyword information for activating voice recognitionis included in the collected voice.

The keyword information, as first voice information stored in the memoryunit 130, is understood as information that a user may input to start avoice recognition service. That is, even when the voice recognitionservice is set to be turned on, the keyword information may be inputtedin order to represent a user's intention of using a voice recognitionservice after the current time.

For example, the keyword information may include pre-mapped informationsuch as “turn on air conditioner” or “voice recognition start”. In sucha manner, at the beginning of using a voice recognition service, byinputting the keyword information, a time for preparation to minimizesurrounding noise or conversation is provided to a user.

When it is recognized that the keyword information is included in avoice command inputted through the voice input unit 110, it isrecognized whether a user truly has an intention for voice command.

On the other hand, when it is recognized that the keyword information isnot included in the voice command, a request message for re-inputtingkeyword information may be outputted from the output unit 160 of the airconditioner 10.

Whether a user truly has an intention for voice command may bedetermined based on whether a user's visage or face is detected for morethan a setting time through the camera 50. For example, the setting timemay be 2 sec to 3 sec. When a user stares at the camera 50 at the frontof the air conditioner 10, the camera 50 captures the user's visage orface and transmits it to the face detection unit 180.

Then, the face detection unit 180 may recognize whether a user stares atthe camera 50 for a setting time. When the face detection unit 180recognizes the user's staring state, the main control unit 120determines that the user has an intention on voice command and enters avoice recognition standby state in operations S14 and S15.

As entering the voice recognition standby state, a filtering process isperformed on all voice information collected through the input unit 110of the air conditioner 10 and then, whether there is a voice command isrecognized. The termination of the voice recognition standby state maybe performed when a user inputs keyword information for voicerecognition termination as voice or manipulates an additional input unitin operation S16.

In such a way, since whether to activate a voice recognition service isdetermined by simply recognizing keyword information according to avoice input or detecting a user's visage or face, voice misrecognitionmay be prevented.

Although it is described with reference to FIG. 5 that the user facedetection operation is performed after the voice keyword informationrecognition operation, unlike this, after the user face detectionoperation, the voice keyword information recognition operation may beperformed.

As another example, although it is described with reference to FIG. 5that when two conditions on the voice keyword information recognitionand the user face detection are satisfied, the voice recognition standbyis entered, when any one condition on the voice keyword informationrecognition and the user face detection is satisfied, the voicerecognition standby may be entered.

Hereinafter, a second embodiment of the present invention is described.In this embodiment, since there is a difference from the firstembodiment in that a voice recognition service is performed through amobile device, the difference is mainly described and for the same partsas in the first embodiment, the description and reference numbers of thefirst embodiment are incorporated.

FIG. 6 is a schematic view illustrating a configuration of a voicerecognition system according to a second embodiment of the presentinvention.

Referring to FIG. 6, the voice recognition system includes a mobiledevice 200 receiving a user's voice input and an air conditioner 10operating and controlled based on a voice inputted to the mobile device200. The air conditioner 10 is just one example of a smart homeappliance. Thus, the idea of this embodiment is applicable to othersmart home appliances. The mobile device 200 may include a smartphone, aremote controller, and a tap book.

The mobile device 200 may include a voice input available voice inputunit 220, a manipulation available input unit 210, a display unitequipped at the front part to display information on an operation stateof the mobile device 200 or information provided from the mobile device200, and a movement detection unit 230 detecting a movement of themobile device 200.

The voice input unit 220 may include a mike. The voice input unit 220may be disposed at a lower part of the mobile device 200. The input unit210 may include a user press manipulation available button or a usertouch manipulation available touch panel.

Then, the movement detection unit 230 may include an acceleration sensoror a gyro sensor. The acceleration sensor or the gyro sensor may detectinformation on an inclined angle of the mobile device 200, for example,an inclined angle with respect to the ground.

For example, when a user stands up the mobile device 200 in order tostare at the display unit 260 of the mobile device 200 and a users laysdown the mobile terminal 200 in order to input a voice command throughthe voice input unit 220, the acceleration sensor or the gyro sensor maydetect a different angle value according to the standing degree of themobile device 200.

As another example, the movement detection unit 230 may include anillumination sensor. The illumination sensor may detect the intensity ofexternal light collected according the standing degree of the mobiledevice 200. For example, when a user stands up the mobile device 200 inorder to stare at the display unit 260 of the mobile device 200 and ausers lays down the mobile terminal 200 in order to input a voicecommand through the voice input unit 220, the acceleration sensor or thegyro sensor may detect a different angle value according to the standingdegree of the mobile device 200.

The mobile device 200 further includes a control unit 250 receivinginformation detected by the input unit 210, the voice input unit 230,the display unit 260, and the movement detection unit 230 andrecognizing a user's voice command input intention, and a communicationmodule 270 communicating with the air conditioner 10.

For example, a communication module of the air conditioner 10 and thecommunication module 270 of the mobile device 100 may directlycommunicate with each other. That is, direct communication is possiblewithout going through a wireless access point by using Wi-Fi-Directtechnique, an Ad-Hoc mode (or network), or Bluetooth.

In more detail, WiFi-Direct may mean a technique for communicating athigh speed by using communication standards such as 802.11a, b, g, and nregardless of the installation of an access point. This technique isunderstood as a communication technique for connecting the airconditioner 10 and the mobile device 200 by using Wi-Fi wirelesslywithout internet network.

The Ad-Hoc mode (or Ad-Hoc network) is a communication network includingonly mobile hosts without a fixed wired network. Since there are nolimitations in the movement of a host and it does not require a wirednetwork and a base station, fast network configuration is possible andits cost is inexpensive. That is, wireless communication is possiblewithout the access point. Accordingly, in the Ad-Hoc mode, without theaccess point, wireless communication is possible between the airconditioner 10 and the mobile device 200.

In the Bluetooth communication, as a short-range wireless communicationmethod, wireless communication is possible within a specific rangethrough a pairing process between a communication module (a firstBluetooth module) of the air conditioner 10 and the communication module270 (a second Bluetooth module) of the mobile device.

As another example, the air conditioner 10 and the mobile device 200 maycommunicate with each other through an access point and a server (notshown) or a wired network. When a user's voice command intention isrecognized, the control unit 250 may transmit to the air conditioner 10information that the voice recognition service standby state is enteredthrough the communication module 270.

FIG. 8 is a flowchart illustrating a control method of a smart homeappliance according to a second embodiment of the present invention.FIG. 9 is a view when a user performs an action for starting a voicerecognition by using a mobile device according to a second embodiment ofthe present invention.

Referring to FIG. 8, in controlling a smart home appliance according tothe second embodiment of the present invention, a voice recognitionservice may be set to be turned on in operation S21.

Then, a user may input or speak a command for a voice recognitionstandby preparation state through a manipulation of the input unit 210or the voice input unit 220. When an input of the input unit 210 isrecognized or it is recognized that keyword information for voicerecognition standby is included in a voice collected through the voiceinput unit 220, it is determined that “voice recognition standbypreparation state” is entered.

As described in the first embodiment, the keyword information, as firstvoice information stored in the memory unit 130, is understood asinformation that a user may input to start a voice recognition servicein operations S22 and S23.

When it is recognized that “the voice recognition standby preparationstate” is entered, whether a user truly has an intention on voicecommand is recognized. On the other hand, when it is not recognized that“the voice recognition standby preparation state” is entered, operationS22 and the following operations may be performed again.

Whether a user truly has an intention on voice command may be determinedbased on whether a detection value is changed in the movement detectionunit 230. For example, when the movement detection unit 230 includes anacceleration sensor or a gyro sensor, whether a value detected by theacceleration sensor or the gyro sensor is changed is recognized. Thechange may depend on whether an inclined value (or range) at which themobile device 200 stands changes into an inclined value (or range) atwhich the mobile device 200 lies.

As shown in FIG. 9, while a user stares at the display unit 260 asgripping the mobile device 200, the mobile device 200 may stand upsomewhat. At this point, an angle at which the mobile device 200 makeswith the ground may be al.

On the other hand, when a user inputs a predetermined voice commandthrough the voice input unit 220 disposed at a lower part of the mobiledevice 200 as gripping the mobile device 200, the mobile device 200 maylie somewhat. At this point, an angle at which the mobile device 200makes with the ground may be α2. Then, α1>α2. Values for α1 and α2 maybe predetermined within a predetermined setting range.

When a value detected by the acceleration sensor or the gyro sensorchanges from a setting range of α1 into a setting range of α2, it isrecognized that a users puts the voice input unit 220 close to a user'smouth. In this case, it is recognized that the user has an intention onvoice command.

As another example, when the movement detection unit 230 includes anillumination sensor, it is recognized whether a value detected by theillumination sensor is changed. The change may depend on whether theintensity of light (first intensity) collected when the mobile device200 stands changes into the intensity of light (second intensity)collected when the mobile device 200 lies.

Herein, the second intensity may be formed greater than the firstintensity. That is, the intensity of light collected from the outsidewhen the mobile device 200 lines may be greater than the intensity oflight collected from the outside when the mobile device 200 stands.Herein, values for the first intensity and the second intensity may bepredetermined within a predetermined setting range.

When it is detected that a value detected by the illumination sensorchanges from a setting range of the first intensity into a setting rangeof the second intensity, it is determines that a user puts the voiceinput unit 220 closer to the user's mouth. In this case, it isrecognized that the user has an intention on voice command in operationsS24 and S25.

Through such a method, when it is recognized that the user has anintention on voice command, a voice recognition standby state may beentered. As entering the voice recognition standby state, a filteringprocess is performed on all voice information collected through thevoice input unit 220 of the air conditioner 10 and then, whether thereis a voice command is recognized.

The termination of the voice recognition standby state may be performedwhen a user inputs keyword information for voice recognition terminationas voice or manipulates an additional input unit. In such a way, sincewhether keyword information according to a simple voice input isrecognized and whether to activate a voice recognition service isdetermined by detecting a movement of a mobile device, voicemisrecognition may be prevented in operation S26.

Although it is described with reference to FIG. 8 that the movementdetection operation of the mobile device is performed after the user'sbutton (touch) input or voice input operation, the user's button (touch)input or voice input operation may be performed after the movementdetection operation of the mobile device.

As another example, although it is descried with reference to FIG. 8that when two conditions on the user's button (touch) input or voiceinput operation and the movement detection operation of the mobiledevice are satisfied, a voice recognition standby is entered, when anyone condition on the user's button (touch) input or voice inputoperation and the movement detection operation of the mobile device issatisfied, the voice recognition standby state may be entered.

FIG. 10 is a view illustrating a configuration of a plurality of smarthome appliances according to a third embodiment of the presentinvention. FIG. 11 is a view when a user makes a voice on a plurality ofsmart home appliances according to a third embodiment of the presentinvention.

Referring to FIG. 10, a voice recognition system 10 according to thethird embodiment of the present invention includes a plurality of voicerecognition available smart home appliances 310, 320, 330, and 340. Forexample, the plurality of smart home appliances 310, 320, 330, and 340may include a cleaner 310, a cooker 320, an air conditioner 330, and arefrigerator 340.

The plurality of smart home appliances 310, 320, 330, and 340 may be ina standby state for receiving a voice. The standby state may be enteredwhen a user sets a voice recognition mode in each smart home appliance.Then, the setting of the voice recognition mode may be accomplished byan input of a predetermined input unit or an input of a set voice.

The plurality of smart home appliances 310, 320, 330, and 340 may bedisposed together in a predetermined space. In this case, even when auser speaks a predetermined voice command toward a specific one amongthe plurality of smart home appliances 310, 320, 330, and 340, anotherhome appliance may react to the voice command. Accordingly, thisembodiment is characterized in that when a user makes a predeterminedvoice, a target home appliance to be commanded is estimated ordetermined appropriately.

In more detail, referring to FIG. 11, each of the smart home appliances310, 320, 330, and 340 includes a voice input unit 510, a voicerecognition unit 520, and a command recognition unit 530.

The voice input unit 510 may collects voices that a user makes. Forexample, the voice input unit 510 may include a microphone. The voicerecognition unit 520 extracts a text from the collected voice. Thecommand recognition unit 530 determines whether there is a text where aspecific word related to an operation of each home appliance is used byusing the extracted text. The command recognition unit 530 may include amemory storing information related to the specific word.

If a voice in which the specific word is used is included in thecollected voice, the command recognition unit 530 may recognize that acorresponding home appliance is a home appliance that is a user'scommand target. The voice recognition unit 520 and the commandrecognition unit 530 are functionally distinguished and described butmay be equipped inside one controller.

A home appliance recognized as a command target may output a messagethat whether a user's command target is the home appliance itself. Forexample, when a home appliance recognized as a command target is the airconditioner 330, a voice or text message “turn on air conditioner?” maybe outputted. At this, the outputted voice or text message is referredto as “recognition message”.

According thereto, when an operation of the air conditioner 330 isdesired, a user may input a recognition or confirmation message that anair conditioner is a target, for example, a concise message “airconditioner operation” or “OK”. At this, the inputted voice message isreferred to as “confirmation message”.

On the other hand, if a voice in which the specific word is used is notincluded in the collected voice, the command recognition unit 530 mayrecognize that corresponding home appliances, that is, the cleaner 310,the cooker 320, and the refrigerator 340, are excluded from a user'scommand target. Then, even when a user's voice is inputted for a settingtime after the recognition, the home appliances excluded from thecommand target are not recognized as the user's command target and donot react to the user's voice.

It is shown in FIG. 11 that when it is recognized that a voicecorresponding to the air conditioner 330 among a plurality of homeappliances is inputted, a recognition message is outputted and anoperation in response to an inputted command is performed.

Moreover, when a home appliance recognized as the command target is morethan one, as mentioned above, each home appliance may output aconfirmation message that whether a user's command target is each homeappliance itself. Then, a user may specify a home appliance that is acommand target by inputting a voice for the type of a home appliance tobe commanded among a plurality of home appliances.

For example, when there are the voice recognition available cleaner 310,cooker 320, air conditioner 330, and refrigerator 340 together in home,as a user makes a voice “air conditioning start”, the air conditioner330 recognizes that a specific word “air conditioning” is used and alsorecognizes that the home appliance itself is a command target. Ofcourse, information on the text “air conditioning” may be stored in thememory of the air conditioner 330 in advance.

On the other hand, since the word “air conditioning” is not a specificword of a corresponding home appliance with respect to the cleaner 310,the cooker 320, and the air conditioner 330, it is recognized that thehome appliances 310, 320, and 330 are excluded from the command target.

As another example, when a user makes the voice “temperature up”, thecooker 320, the air conditioner 330, and the refrigerator 340 mayrecognize that a specific word “temperature” is used. That is, theplurality of home appliances 320, 330, and 340 may recognize that theyare the command targets.

At this point, the plurality of home appliances 320, 330, and 340 mayoutput a message that whether the user's command targets are the homeappliances 320, 330, and 340 themselves. Then, as a user inputs a voicefor a specific home appliance, for example, “air conditioner”, it isspecified that a command target is the air conditioner 330. When acommand target is specified in the above manner, an operation of a homeappliance may be controlled through an interactive communication betweena user and a corresponding home appliance.

In such a manner, when there are a plurality of voice recognitionavailable smart home appliances, a command subject may be recognized byextracting the feature (specific word) of a voice that a user makes andonly a specific electronic product among a plurality of electronicproducts responds according to the recognized command subject.Therefore, miscommunication may be prevented during an operation of anelectronic product.

FIG. 12 is a view when a plurality of smart home appliances operate byusing a mobile device according to a fourth embodiment of the presentinvention.

Referring to FIG. 12, a voice recognition system 10 according to thefourth embodiment of the present invention includes a mobile device 400receiving a user's voice input, a plurality of home appliances 310, 320,330, and 340 operating and controlled based on a voice inputted to themobile device 400, and a server 450 communicably connecting the mobiledevice 400 and the plurality of home appliances 310, 320, 330, and 340.

The mobile device 400 is equipped with the voice input unit 510described with reference to FIG. 11 and the server 450 includes thevoice recognition unit 520 and the command recognition unit 530.

The mobile device 400 may include an application connected to the server400. Once the application is executed, a voice input mode for user'svoice input may be activated in the mobile device 400.

When a user's voice is inputted through the voice input unit 510 of themobile device 400, the inputted voice is delivered to the server 450 andthe server 450 determines which home appliance is the target of a voicecommand as the voice recognition unit 520 and the command recognitionunit 530 operate.

When a specific home appliance is recognized as the command target onthe basis of a determination result, the server 450 notifies thespecific home appliance of the recognized result. The home appliancenotified of the result responses to a user's command. For example, whenthe air conditioner 330 is recognized as a command target and notifiedof a result, it may output a recognition message such as “turn on airconditioner?”. According thereto, a user may input a confirmationmessage such as “OK” or “air conditioner operation”. In relation tothis, the contents described with reference to FIG. 11 are used.

FIG. 13 is a view illustrating a configuration of a smart home applianceor a mobile device and an operating method thereof according to anembodiment of the present invention. Configurations shown in FIG. 13 maybe equipped in smart home appliances or mobile devices. Hereinafter,smart home appliances will be described for an example.

Referring to FIG. 13, a smart home appliance according to an embodimentof the present invention includes a voice input unit 510 receiving auser's voice input and a voice recognition unit 520 extracting a textfrom a voice collected through the voice input unit 510. The voicerecognition unit 520 may include a memory unit where the frequency of avoice and a text are mapped.

The smart home appliance may further include a region recognition unit540 extracting the intonation of a voice inputted from the voice inputunit 510 to determine the local color of the voice, that is, whichregion dialect is used. The region recognition unit 540 may include adatabase for dialects used in a plurality of regions. The database maystore information on the intonation recognized when speaking in dialect,that is, unique frequency changes.

The text extracted through the voice recognition unit 520 and theinformation on a region determined through the region recognition unit540 may be delivered to the control unit 550.

The smart home appliance may further include a memory unit 560 mappingthe text extracted by the voice recognition unit 520 and a functioncorresponding to the text and storing them.

The control unit 550 may recognize a function corresponding to the textextracted by the voice recognition unit 520 on the basis of theinformation stored in the memory unit 560. Then, the control unit 550may control a driving unit 590 equipped in the home appliance in orderto perform the recognized function.

For example, the driving unit 560 may include a suction motor of acleaner, a motor or a heater of a cooker, a compressor motor of an airconditioner, or a compressor motor of a refrigerator.

The home appliance further includes a display unit 570 outputting orproviding region customized information to a screen or a voice outputunit 580 outputting a voice on the basis of a setting functioncorresponding to the text extracted by the voice recognition unit 520and the region information determined by the region recognition unit540. The combined display unit 570 and voice output unit 580 may bereferred to as “output unit”

That is, the setting function may include a plurality of functionsdivided according to regions and one function matching the regioninformation determined by the region recognition unit 540 among theplurality of functions may be outputted. In summary, combinedinformation of the recognized function and the determined local colormay be inputted to the display unit 570 or the voice output unit 580(region customized information providing service).

Moreover, the smart home appliance further includes a selectionavailable mode setting unit 565 to perform a mode for the regioncustomized information providing service. A user may use the regioncustomized information providing service when the mode setting unit 565is in “ON” state. Of course, a user may not use the region customizedinformation providing service when the mode setting unit 565 is in “OFF”state.

Hereinafter, by referring to the drawings, contents of region customizedinformation outputted to the display unit 570 are described.

FIG. 14 is a view illustrating a message output of a display unitaccording to an embodiment of the present invention.

Referring to FIG. 14, the display unit 570 may be equipped in the cooler320, the refrigerator 340, or the mobile device 400. Hereinafter, thedisplay unit 570 equipped at the refrigerator 340 is described for anexample.

The cooler 320 or the refrigerator 340 may provide information on arecipe for a predetermined cooking to a user. In other words, the cooker320 or the refrigerator 340 may include a memory unit storing recipeinformation on at least one cooking.

When a region customized information providing service is used to obtainrecipe information, a user may provide an input when the mode settingunit 565 of the refrigerator 340 is ON state.

When the region customized information providing service starts, forexample, a guide message such as “what can I help you”, that is, a voiceinput request message, may be displayed on a screen of the display unit570. Of course, the voice input request message may be outputted as avoice through the voice output unit 580.

As shown in FIG. 14, a user may input a specific recipe for the voiceinput request message, for example, as shown in FIG. 14, a voice “beefradish soup recipe”. The refrigerator 340 receives and recognizes auser's voice command and extracts a text corresponding to the recognizedvoice command. Accordingly, the display unit 570 may display a screen“beef radish soup recipe”.

Then, the refrigerator 340 extracts the intonation from a voice inputtedby a user and recognizes a frequency change corresponding to theextracted intonation, so that it may recognize a dialect for a specificregion.

For example, when a user inputs a voice “beef radish soup recipe” inGyeongsang-do accent, the refrigerator 340 recognizes the Gyeongsang-dodialect and prepares to provide a recipe optimized for Gyeongsang-doregion. That is, there are a plurality of beef radish recipes accordingto regions, one recipe matching the recognized region, Gyeongsang-do,may be recommended.

As a result, the refrigerator 340 may recognize that a user inGyeongsang-do region wants to receive ““beef radish soup recipe” and maythen, read information on a Gyeongsang-do style radish recipe to provideit to a user. For example, the display unit 570 may display a message“here is Gyeongsang-do style red beef radish soup recipe”. In addition,a voice message may be outputted through the voice output unit 580.

Another example is described.

When the smart home appliance is an air conditioner for conditioning anindoor space and recognizes that a user′ region is a cold region such asGangwon-do, as a user inputs a voice command “temperature down”, underthe assumption that a user in cold region likes cold weather, the smarthome appliance may operate to set a relatively low temperature as asetting temperature. Then, information on contents related to adjustinga setting temperature to a relatively low temperature, for example, 20°C., may be outputted to the output units 570 and 580.

According to such a configuration, without a user's input for specificinformation, the dialect that a user speaks is recognized and regioncustomized information is provided on the basis of the recognizeddialect information. Therefore, usability may be improved.

FIG. 15 is a view illustrating a message output of a display unitaccording to another embodiment of the present invention.

Referring to FIG. 15, the display unit 570 according to anotherembodiment of the present invention may be equipped in the airconditioner 330 or the mobile device 400. When a region customizedinformation providing service is used to input a command for anoperation of the air conditioner 330, a user may provide an input whenthe mode setting unit 545 of the air conditioner 330 is in ON state.

When the region customized information providing service starts, forexample, a guide message such as “what can I help you”, that is, a voiceinput request message, may be displayed on a screen of the display unit570. Of course, the voice input request message may be outputted as avoice through the voice output unit 580.

With respect to the voice input request message, a user may input acommand on an operation of the air conditioner 300, for example, asshown in FIG. 15, a voice “turn on air conditioner (in dialect)”. Theair conditioner 330 receives and recognizes a user's voice command andextracts a text corresponding to the recognized voice command.Accordingly, the display unit 570 may display a screen “turn on airconditioner (in dialect)”.

Then, the air conditioner 330 extracts the intonation from a voiceinputted by a user and recognizes a frequency change corresponding tothe extracted intonation, so that it may recognize a dialect for aspecific region. For example, when a user inputs a voice “turn on airconditioner” in Jeolla-do accent, the air conditioner 330 may recognizethe Jeolla-do dialect and may then generate a response message for auser as the Jeolla-do dialect.

That is, the air conditioner 330 recognizes that a user in Jeolla-doregion wants “air conditioner operation” and reads dialect informationon a message that an air conditioner operation is performed from thememory unit 560 to provide it to a user. For example, the display unit570 may output a message using the Jeolla-do dialect, for example, “itis very hot and turn on quickly (in the Jeolla-do dialect)”. Inaddition, a voice message may be outputted through the voice output unit580.

By such a configuration, the dialect that a user speaks is recognizedand information to be provided to a user is provided as a dialect on thebasis of the recognized dialect information, so that the user may feelthe intimacy.

FIGS. 16A and 16B are views illustrating a security setting for voicerecognition function performance according to a fifth embodiment of thepresent invention.

Referring to FIGS. 16A and 16B, a user's security setting is possible inthe voice recognition system according to the fifth embodiment of thepresent invention. The security setting may be completed by a smart homeappliance directly or by using a mobile device. Hereinafter, forexample, security setting and authentication procedures by using amobile device are described.

When a user wants to use the region customized information providingservice, an input may be provided when the mode setting unit 565 in themobile device 400 is in ON state. As an input is provided when the modesetting unit 565 is in ON state, an operation for setting an initialsecurity may be performed.

The mobile device 400 may output a message for a predetermined key word.For example, as shown in the drawing, a key word “calendar” may beoutputted as a text through a screen of the mobile device 400 or may beoutputted as voice through a speaker.

Then, together with a message for the outputted key word, a first guidemessage may be outputted. The first guide message includes contents foran input of a reply word to the key word. For example, the first guidemessage may include content “please speak word coming to mind whenlooking at the next word”. Then, the first guide message may beoutputted as a text through a screen of the mobile device 400 or may beoutputted as a voice through a speaker.

For the first guide message, a user may input a word to be set as apassword as a voice. For example, as shown in the drawing, a relay word“cat” may be inputted. When the mobile device 400 recognizes the replyword, a second guide message notifying the reply word is stored may beoutputted through a screen or a voice.

In such way, when the region customized information providing service isused after the completion of the security setting, as shown in FIG. 16B,a procedure for performing an authentication by inputting a reply wordto the key word may be performed.

In more detail, when an input is provided as the mode setting unit 565in the mobile device 400 is in ON state, the mobile device 400 outputs amessage for the key word, for example, “calendar”, and outputs a thirdguide message notifying the need for authentication, for example, “userauthentication is required for this function”. The message for keywordand the third guide message may be outputted through a screen of themobile device 400 or a voice.

For the key word, a user may input a predetermined set reply word, forexample, a voice of “cat”. As recognizing the matching of the key wordand the reply word, the mobile device 400 may output a fourth guidemessage notifying that authentication is successful, for example, a textof voice message “authenticated”.

In such a way, after the security is set to use the region customizedinformation providing service, a predetermined reply word is configuredto be inputted in a usage stage, so that other users' service access andusage are limited except for designated users.

FIG. 17 is a view illustrating a configuration of a voice recognitionsystem and its operation method according to a sixth embodiment of thepresent invention.

Referring to FIG. 17, a smart home appliance according to the sixthembodiment of the present invention includes a voice input unit 510receiving a user's voice input and a voice recognition unit 520extracting a language element as a text from voice information collectedthrough the voice input unit 510. The voice recognition unit 520 mayinclude a memory unit where the frequency of a voice and a text aremapped.

The smart home appliance may further include an emotion recognition unit540 extracting user's emotion information from the voice informationinputted through the voice input unit 510. The emotion recognition unit540 may include a database where information on user's voicecharacteristics and information on an emotion state are mapped. Theinformation on user's voice characteristics may include information onspeech spectrum having distinctive characteristics for each user'semotion.

The speech spectrum represents a distribution according to a voice'sfrequency and may be understood as that a frequency distribution foreach emotion, that is, emotions such as joy, angry, and sadness, ispatterned. Accordingly, when a user makes a voice with a predeterminedemotion, the emotion recognition unit 540 interprets a frequency changeto extract a user's emotion.

The text extracted through the voice recognition unit 520 and theinformation on an emotion determined through the emotion recognitionunit 540 may be delivered to the control unit 550. The smart homeappliance may further include a memory unit 560 mapping the textextracted by the voice recognition unit 520 and a function correspondingto the text and storing them.

The control unit 550 may recognize a function corresponding to the textextracted by the voice recognition unit 520 on the basis of theinformation stored in the memory unit 560. Then, the control unit 550may control a driving unit 590 equipped in the home appliance in orderto perform the recognized function.

The home appliance further includes a display unit 570 outputting orproviding user customized information to a screen or a voice output unit580 outputting a voice on the basis of a setting function correspondingto the text extracted by the voice recognition unit 520 and the emotioninformation extracted by the region recognition unit 540.

That is, the setting function may include a plurality of functionsdivided according to user's emotions and one function matching theemotion information determined by the emotion recognition unit 540 amongthe plurality of functions may be outputted. In summary, the displayunit 570 or the voice output unit 580 may output combined information ofa function corresponding to the text and user's emotion information(user customized information providing service).

Moreover, the smart home appliance further includes a selectionavailable mode setting unit 565 to perform a mode for the usercustomized information providing service. A user may use the usercustomized information providing service when the mode setting unit 565is in “ON” state. Of course, a user may not use the user customizedinformation providing service when the mode setting unit 565 is in “OFF”state.

FIGS. 18 to 20 are views illustrating a message output of a display unitaccording to a sixth embodiment of the present invention. A display unit570 shown in FIGS. 18 to 20 may be equipped in the air conditioner 330or the mobile device 450. Hereinafter, the display unit 570 equipped atthe air conditioner 330 is described for an example.

First, referring to FIG. 18, when a user customized informationproviding service is used by using the air conditioner 330 to conditionthe air in an indoor space, a user may provide an input when the modesetting unit 565 is in ON state.

When the user customized information providing service starts, forexample, a guide message such as “what can I help you”, that is, a voiceinput request message, may be displayed on a screen of the display unit570. Of course, the voice input request message may be outputted as avoice through the voice output unit 580.

As shown in FIG. 18, a user may input a specific operation command forthe voice input request message, for example, as shown in FIG. 18, avoice “air conditioner start”. The air conditioner 330 receives andrecognizes a user's voice command and extracts a text (first voiceinformation) corresponding to the recognized voice command. Accordingly,the display unit 570 may display a screen “air conditioner start”.

Then, the air conditioner 330 may extract emotion information (secondvoice information) from a voice inputted from a user by operating theemotion recognition unit 540. In more detail, information on a frequencychange detected from a user's voice may be compared to information on aspeech spectrum having characteristics for people's each emotion. Then,information corresponding to each other may be matched based on acomparison result and accordingly, emotion information that a user'svoice has may be obtained.

For example, when a user inputs “air conditioner start” with an angryvoice, the emotion recognition unit 540 may recognize that a user makesa voice with an angry emotion from a frequency change detected from theuser's voice. The air conditioner 330 combines the first voiceinformation and the second voice information to recognize a functionthat a user wants and recommend a predetermined function matching auser's emotion, that is, a specific operation mode.

That is, as shown in FIG. 18, as an operation of the air conditioner 330starts according to a user's command, an operation mode is recommendedin consideration of an emotion annoyed with hot weather. For example,the display unit 570 may output a message “oh! very hot? air conditionerstart with lowest temperature and direct wind?”

Herein, the direct wind is understood as a mode in which the dischargedirection of air is formed directly toward the position of a userdetected through the body detection unit 36 of the air conditioner 330.That is, a setting temperature is set to the lowest temperature toperform an air conditioning function and cool wind reaches a userdirectly, so that a user may feed cool instantly. When a voice that auser accepts or selects the outputted message, for example, “yes”, isinputted, the air conditioner 330 recognizes this and operates in therecommended mode.

It is described with reference to FIG. 18 that after the air conditioner330 recommends a specific mode to a user and a user's recommendationmode acceptance is recognized, the air conditioner 330 operatesaccording to the specific mode. However, unlike this, the recommendedmode may operate instantly.

Then, referring to FIG. 19, when a user customized information providingservice starts and a voice input request message is outputted to ascreen of the display unit 570 or as a voice, a user may input aspecific command for the voice input request message, for example, avoice “temperature up”

The air conditioner 330 receives and recognizes a user's voice commandand extracts a text (first voice information) corresponding to therecognized voice command. Accordingly, the display unit 570 may displaya screen “temperature up”.

Then, the air conditioner 300 may extract emotion information (secondvoice information) from a voice inputted from a user by operating theemotion recognition unit 540. For example, when a user inputs“temperature up” with a trembling voice, the emotion recognition unit540 may recognize that a user makes a voice with a trembling voice dueto cold from a frequency change detected from the user's voice.

The air conditioner 330 combines the first voice information and thesecond voice information to recognize a function that a user wants andrecommend a predetermined function matching a user's emotion, that is, aspecific operation mode. That is, as shown in FIG. 19, as an operationof the air conditioner 330 starts according to a user's command, anoperation mode is recommended in consideration of a cold tremblingstate. For example, the display unit 570 may output a message “cold? setair conditioner temperature to 26 degrees. execute action detectionindirect wind?”.

Herein, the indirect wind is understood as a mode in which the dischargedirection of air is indirectly provided toward a user by avoiding theposition of the user detected through the body detection unit 36 (seeFIG. 2) of the air conditioner 300. That is, a setting temperature risesto a relatively high temperature to perform a cooling or heatingfunction and wind reaches a user indirectly, so that the user may feelpleasant without cold. When a voice that a user accepts the outputtedmessage, for example, “yes”, is inputted, the air conditioner 330recognizes this and operates in the recommended mode.

Then, referring to FIG. 20, when a user customized information providingservice starts and a voice input request message is outputted to ascreen of the display unit 570 or as a voice, a user may input aspecific command for the voice input request message, for example, avoice “air conditioner start”

The air conditioner 330 receives and recognizes a user's voice commandand extracts a text (first voice information) corresponding to therecognized voice command. Accordingly, the display unit 570 may displaya screen “air conditioner start”.

Then, the air conditioner 330 may extract emotion information (secondvoice information) from a voice inputted from a user by operating theemotion recognition unit 540. For example, when a user input “airconditioner start” with a sad voice, the emotion recognition unit 540may recognize that a user makes a voice with a sad emotion from afrequency change detected from the user's voice.

The air conditioner 330 combines the first voice information and thesecond voice information to recognize a function that a user wants andrecommend a predetermined function matching a user's emotion, that is, aspecific operation mode. That is, as shown in FIG. 20, as an operationof the air conditioner 330 starts according to a user's command, anoperation mode is recommended in consideration of a user's sad emotionstate. For example, the display unit 570 may output a message “airconditioner start. sir, use aroma function for refresh. start aromafunction?”.

Herein, the aroma function is understood as a function through which acapsule inserted into the capsule injection device 60 (see FIG. 2) ofthe air conditioner 300 acts so that a wind with an aroma fragrance isdischarged. That is, the air conditioner 300 recognizes a user's sademotion and then diffuses an aroma fragrance for refresh into an indoorspace. When a voice that a user accepts the outputted message, forexample, “yes”, is inputted, the air conditioner 300 recognizes this andoperates in the recommended mode.

FIGS. 21A to 23 are views illustrating a message output of a displayunit according to another embodiment of the present invention. A displayunit 570 shown in FIGS. 21A to 23 may be equipped in the cooker 320, therefrigerator 340, or the mobile device 400. Hereinafter, the displayunit 570 equipped at the refrigerator 340 is described for an example.

First, referring to FIG. 21A, when a user customized informationproviding service starts and a voice input request message is outputtedto a screen of the display unit 570 or as a voice, a user may input aspecific command for the voice input request message, for example, avoice “recipe search”

The refrigerator 340 receives and recognizes a user's voice command andextracts a text (first voice information) corresponding to therecognized voice command. Accordingly, the display unit 570 may displaya screen “recipe search”. Then, the refrigerator 340 may extract emotioninformation (second voice information) from a voice inputted from a userby operating the emotion recognition unit 540.

For example, when a user inputs “recipe search” with a sad voice, theemotion recognition unit 540 may recognize that a user makes a voicewith a sad emotion from a frequency change detected from the user'svoice.

The refrigerator 340 combines the first voice information and the secondvoice information to recognize a function that a user wants andrecommend a predetermined function matching a user's emotion, that is, aspecific recipe. That is, as shown in FIG. 21A, as a recipe is searchedaccording to a user's command, one of a plurality of recipes isrecommended in consideration of a sad emotion. For example, the displayunit 570 may output a message “feel depressed? eat sweet food then youfeel better. sweet food recipe search?” When a user inputs theacceptance voice for the outputted message, for example, “yes”, therefrigerator 340 recognizes this and recommends a specific recipe whileoutputting a message “sweet food recipe search”.

FIG. 21B is similar to FIG. 21A in terms of scenario but when a userrejects a specific recipe that the refrigerator 340 recommends, forexample, when a user inputs a voice rejecting a message “sweet foodrecipe search?” outputted by the display unit 570, that is, “no”, avoice input request message “speak food ingredients to search” may beoutputted.

As shown in FIG. 21B, a voice input request message “what can I helpyou?” may be defined as “first message” and a voice input requestmessage “tell food ingredients to search” may be defined as “secondmessage”. A user may input a voice for another selectable function withrespect to the second message, that is, another food ingredient, toreceive information on a desired recipe.

Then, referring to FIG. 22, when a user customized information providingservice starts and a voice input request message is outputted to ascreen of the display unit 570 or as a voice, a user may input aspecific command for the voice input request message, for example, avoice “recipe search”

The refrigerator 340 receives and recognizes a user's voice command andextracts a text (first voice information) corresponding to therecognized voice command. Accordingly, the display unit 570 may displaya screen “recipe search”.

Then, the refrigerator 340 may extract emotion information (second voiceinformation) from a voice inputted from a user by operating the emotionrecognition unit 540. For example, when a user input “recipe search”with an angry voice, the emotion recognition unit 540 may recognize thata user makes a voice with an angry emotion from a frequency changedetected from the user's voice.

The refrigerator 340 combines the first voice information and the secondvoice information to recognize a function that a user wants andrecommend a function matching a user's emotion, that is, a specificrecipe. That is, as shown in FIG. 22, as a recipe is searched accordingto a user's command, one of a plurality of recipes is recommended inconsideration of an angry emotion. For example, the display unit 570 mayoutput a message “are you angry with empty stomach? fast cook foodrecipe search?”.

When a user inputs the acceptance voice for the outputted message, forexample, “yes”, the refrigerator 340 recognizes this and recommends aspecific recipe while outputting a message “fast cook food recipesearch”. Of course, in the case of FIG. 22, as described with referenceto FIG. 21B, when a user rejects a recommended recipe, the refrigerator340 may output a second message to receive a desired specific recipefrom a user.

Then, referring to FIG. 23, when a user customized information providingservice starts and a voice input request message is outputted to ascreen of the display unit 570 or as a voice, a user may input aspecific command for the voice input request message, for example, avoice “recipe search”

The refrigerator 340 receives and recognizes a user's voice command andextracts a text (first voice information) corresponding to therecognized voice command. Accordingly, the display unit 570 may displaya screen “recipe search”.

Then, the refrigerator 400 may extract emotion information (second voiceinformation) from a voice inputted from a user by operating the emotionrecognition unit 540. For example, when a user inputs “recipe search”with a happy voice, the emotion recognition unit 540 may recognize thata user makes a voice with a happy emotion from a frequency changedetected from the user's voice.

The refrigerator 340 combines the first voice information and the secondvoice information to recognize a function that a user wants andrecommend a function matching a user's emotion, that is, a specificrecipe. That is, as shown in FIG. 23, as a recipe is searched accordingto a user's command, one of a plurality of recipes is recommended inconsideration of a happy emotion. For example, the display unit 570 mayoutput a message “feel good? make special food. special food recipesearch?”.

When a user inputs the acceptance voice for the outputted message, forexample, “yes”, the refrigerator 340 recognizes this and recommends aspecific recipe while outputting a message “special food recipe search”.Of course, in the case of FIG. 23, as described with reference to FIG.21B, when a user rejects a recommended recipe, the refrigerator 340 mayoutput a second message to receive a desired specific recipe from auser.

In such a wary, since a smart home appliance extracts emotioninformation from a user's voice and recommends a specific functionmatching a user's emotion among a plurality of functions, instead ofextracting a text from a user's voice command and performs a setfunction simply, user's convenience may be increased and productsatisfaction may be improved.

FIG. 24 is a block diagram illustrating a configuration of an airconditioner as one example of a smart home appliance according to aseventh embodiment of the present invention. Hereinafter, although anair conditioner is described as one example of a smart appliance, itshould be clear in advance that the ideas related to a voice recognitionor communication (information offer) procedure except for the uniquesetting functions of an air conditioner may be applied to other smarthome appliances.

Referring to FIG. 24, an air conditioner 600 according to the seventhembodiment of the present invention includes a plurality ofcommunication units 680 and 690 communicating with an external device.The plurality of communication units 680 and 690 include a firstcommunication module 680 communicating with the server 700 and aposition information reception unit 690 receiving information on theposition of the air conditioner 600 from a position informationtransmission unit 695.

The first communication module 680 may communicate with the secondcommunication module 780 of the server 780 in a wired or wirelessmanner. For example, the first communication module 680 of the airconditioner 600 and the second communication module 780 of the server700 may communicate with each other directly or through an access point,or through a wired network. In this embodiment, there is no limitationin a communication method between the first communication module 680 ofthe air conditioner 600 and the second communication module 780 of theserver 700.

Moreover, each of the first communication module 680 of the airconditioner 600 and the second communication module 780 of the server700 may have a unique internet protocol (IP) address. Accordingly, whenthe first communication module 680 and the second communication module780 are communicably connected to each other, the server 700 mayrecognize the installed position or region of the air conditioner 600 byrecognizing the first communication module 680.

The position information reception unit 680 may be a GPS reception unit,for example. Then, the position information transmission unit 695 isconfigured to transmit information on the position of the positioninformation reception unit 690 or the air conditioner 600, to theposition information reception unit 690. For example, the information onposition may include a position coordinate value.

The position information transmission unit 695 may be a GPS satellite ora communication base station. The position information reception unit690 may transmit a predetermined signal to the position informationtransmission unit 695 periodically or at a specific time point and theposition information transmission unit 695 may transmit the informationon position to the position information reception unit 690. In that thefirst communication module 680 and the position information receptionunit 695 recognize the position or region of the air conditioner 600,they may be referred to as a “position information recognition unit”.

The server 700 may further include a server memory 770. The servermemory 770 may store information necessary for an operation of the airconditioner 600, for example, information on the position (region) ofthe air conditioner 600 or the first communication module 680 or weatherinformation corresponding to the position (region). When the firstcommunication module 680 and the second communication module 780 areconnected communicably, information stored in the server memory 770 maybe transmitted to the air conditioner 700.

In such a manner, information on the position of the air conditioner 600may be recognized based on information on the communication address whenthe first communication module 680 and the second communication module780 are connected communicably or information received from the positioninformation reception unit 690.

For example, when the first communication module 680 and the secondcommunication module 780 are not connected communicably, the airconditioner 600 may receive information on the position through theposition information reception unit 690. On the other hand, when theposition information reception 690 is not in ON state, the airconditioner 600 may receive the information on position through acommunication connection of the first and second communication modules680 and 780.

FIG. 25 is a flowchart illustrating a control method of a smart homeappliance according to a seventh embodiment of the present invention.

Referring to FIG. 25, in controlling a smart home appliance according tothe seventh embodiment of the present invention, a position recognitionservice may be set to be turned on. The position recognition service isunderstood as a service in which the installed position or region of theair conditioner 600 is recognized when the first and secondcommunication modules 680 and 780 are communicably connected or theposition information reception unit 690 receives position informationand a function of a smart home appliance is performed based oninformation on the recognized position or region. Then, an applicationfor using the position recognition service is executed in operations S31and S32.

A user's voice command is inputted through the voice input unit 110 (seeFIG. 2). Then, information on the position of the smart home applianceis recognized through the communication connection of the first andsecond communication modules 680 and 780 and the position informationreception unit 690 in operations S33 and S34.

A voice command inputted through the voice input unit 110 may correspondto at least one voice information among a plurality of voice informationstored in the memory unit 130 (see FIG. 3) and the corresponding voiceinformation may be extracted as a text. Then, by using the extractedtext, a predetermined setting function among a plurality of settingfunctions that home appliance performs may be recognized in operationS35.

Then, as the smart home appliance performs the recognized settingfunction, information on the position of the smart home appliance may beconsidered. Then, information on the setting function and information onthe position are combined so that predetermined information may beprovided to a user.

For example, when the smart home appliance is located in a region, alanguage used in a corresponding region, that is, a dialect, may guideinformation on the setting function. As another example, information fora setting function optimized for the position of the smart homeappliance may be guided in operation S36. In relation to this,hereinafter, an example of information on a setting function that asmart home appliance provides is described.

FIG. 26 is a view illustrating a display unit of a smart home appliance.FIG. 26 illustrates a view when information on a setting functioncombined with position information is outputted.

Referring to FIG. 26, a message for requesting a voice input may beoutputted from an output unit 660 of the air conditioner 600. Forexample, a message “what can I help you?” may be outputted from theoutput unit 660. At this point, voice and text messages may be outputtedtogether.

For this, a user may speak a voice command “temperature up”. The spokenvoice is inputted through the voice input unit 110 and filtered and thenis delivered to the main control unit 120 (see FIG. 3). The main controlunit 120 recognizes the filtered voice as predetermined voiceinformation and outputs it as a text.

Then, through a communication connection with the server 700 or on thebasis of information received from the position information receptionunit 690, information on the position of the air conditioner 600 may berecognized. The air conditioner 600 may provide a setting functioncorresponding to the voice command to a user according to the recognizedvoice command information. At this point information on the recognizedposition may be considered.

For example, when the position (region) of the air conditioner 600 is inthe Gyeongsang-do region, information on the setting function may beguided in Gyeongsang-do accent. For example, a message for performing afunction to raise a setting temperature in Gyeongsang-do accent “raisetemperature by one degree?” may be outputted. For this, when a userinputs an acceptance intention, that is, a voice “yes”, the main controlunit 120 may control an operation of the driving unit 140 (see FIG. 3)to raise a setting temperature by 1 degree.

FIG. 27 is a view illustrating a configuration of a cooker as anotherexample of a smart home appliance according to a seventh embodiment ofthe present invention. FIGS. 28A and 28B are views illustrating adisplay unit of the cooker.

Referring to FIG. 27, a cooker 810 according to the seventh embodimentof the present invention includes a voice input unit 812 receiving auser's voice input, an input unit 814 manipulated for a user's commandinput, and an output unit 816 displaying information on an operationstate of the cooker 810. The output unit 816 includes a display unitdisplaying information on a screen and a voice output unit outputting avoice.

The cooker 810 includes the filter unit 115, the memory unit 130, thedriving unit 140, the control units 120 and 150, and the firstcommunication module 680, and the position information reception unit690, all of which are described with reference to FIG. 3. Their detaileddescriptions are omitted.

Referring to FIG. 28A, a message for requesting a voice input may beoutputted from the output unit 816 of the cooker 810. For example, amessage “what can I help you?” may be outputted from the output unit816. At this point, voice and text messages may be outputted together.

For this, a user may speak a voice command “food recipe”. The spokenvoice is inputted through the voice input unit 110 and filtered and thenis delivered to the main control unit 120. The main control unit 120recognizes the filtered voice as predetermined voice information andoutputs it as a text. Then, through a communication connection with theserver 700 or on the basis of information received from the positioninformation reception unit 690, information on the position of thecooker 810 may be recognized.

When the main control unit 120 recognizes a user's voice command, theoutput unit 816 may output a message for requesting an input of detailedinformation on a recipe. For example, a message “please input food type”may be outputted as voice or text. A user may input information on adesired food type, that is, a food keyword, through the input unit 815.For example, a user may input a food keyword “grilled food”.

Once a user's food keyword input is completed, the cooker 810 maycomplete a related recipe search and may then output a guide message.For example, a message “recipe search is completed. want to check searchresult?” may be outputted. For this, when a user inputs an acceptanceintention, that is, a voice “yes”, a screen may be switched and anotherscreen shown in FIG. 28B may be outputted.

Referring to FIG. 28B, information on the position and setting functioninformation corresponding to the recognized voice command may becombined and predetermined information may be outputted to the outputunit 816 of the cooker 810.

In more detail, among recipe information on “grilled food” that a userwants, specialty or traditional food recipes in the position (region) ofthe cooker 810 may be arranged preferentially and outputted to theoutput unit 360. For example, a specialty or traditional food recipe for“grilled food”, that is, “oven-grilled pork roll with fishery”,“assorted grilled seafood”, and “assorted grilled mushroom”, may bearranged at an upper part of the output unit 816 and may be displayed ina check box. That is, among a plurality of information on a settingfunction, information optimized for the position recognized by theposition information recognition units 680 and 690 may be outputtedfirst to the output unit 816.

Then, general grilled food recipes may be arranged at a lower part ofthe specialty or traditional food recipe. When a user selects a desiredrecipe among each arranged recipe, detailed information on a recipe maybe checked. In such a way, since information on a setting function of ahome appliance is provided on the basis of information on a user's voicecommand and position information of a home appliance, user's conveniencemay be increased.

FIG. 29 is a view illustrating a configuration of a washing machine asanother example of a smart home appliance according to an eightembodiment of the present invention. FIG. 30 is a flowchart illustratinga control method of a smart home appliance according to the eighthembodiment of the present invention.

Referring to FIG. 29, the smart home appliance according to the eighthembodiment may include a washing machine 820.

The washing machine 820 includes a voice input unit 822 receiving auser's voice input, an input unit 825 manipulated for a user's commandinput, and an output unit 826 displaying information on an operationstate of the washing machine 820. The output unit 826 includes a displayunit displaying information on a screen and a voice output unitoutputting a voice. The washing machine 820 includes the filter unit115, the memory unit 130, the driving unit 140, the control units 120and 150, and the first communication module 680, and the positioninformation reception unit 690. Their detailed descriptions are omitted.

Referring to FIG. 30, in controlling a smart home appliance, a positionrecognition service may be set to be turned on. The position recognitionservice is understood as a service in which the installed position orregion of the washing machine 820 is recognized when the first andsecond communication modules 680 and 780 are communicably connected orthe position information reception unit 690 receives positioninformation and a function of a smart home appliance is performed basedon information on the recognized position or region. Then, anapplication for using the position recognition service is executed inoperations S41 and S42.

A user's voice command is inputted through the voice input unit 110.Then, information on the position of the smart home appliance isrecognized through the communication connection of the first and secondcommunication modules 680 and 780 and the position information receptionunit 690 in operations S43 and S44. Then, weather information of theposition (region) where the washing machine 820 is installed is receivedfrom the server 700 in operation S45.

A voice command inputted through the voice input unit 110 may correspondto at least one voice information among a plurality of voice informationstored in the memory unit 130 and the corresponding voice informationmay be extracted as a text. Then, by using the extracted text, apredetermined setting function among a plurality of setting functionsthat home appliance performs may be recognized in operation S46.

Then, as the smart home appliance performs the recognized settingfunction, weather information on the installed region of the smart homeappliance may be considered. Then, information on the setting functionand information on the weather are combined so that recommendationinformation related to the setting function may be provided to a user.That is, one information among a plurality of information related to thesetting function may be recommended.

For example, when a smart home appliance is the washing machine 820 anda voice command is “laundry start”, a laundry course may be recommendedby recognizing weather information on a region where the washing machine820 is installed. For example, if rainy or high humidity, strong spin ora drying function may be recommended in operation S47.

As another example, when a smart home appliance is an air conditioner,in the case of a voice command “turn on air conditioner”, a drivingcourse may be recommended by recognizing weather information on a regionwhere the air conditioner is installed. For example, a dehumidifyingfunction may be recommended by receiving humidity information. Then, auser sets a bedtime reservation, by receiving a temperature of thenighttime, a recommendation for increasing or decreasing a reservationtime.

FIG. 31 is a block diagram illustrating a configuration of a voicerecognition system according to a ninth embodiment of the presentinvention.

Referring to FIG. 31, a voice recognition system according to the ninthembodiment of the present invention includes a mobile device 900receiving a user's voice input, a plurality of home appliances 810, 820,830, and 840 operating and controlled based on a voice inputted to themobile device 900, and a server 950 communicably connecting the mobiledevice 900 and the plurality of home appliances 810, 820, 830, and 840.

For example, the plurality of smart home appliances 810, 820, 830, and840 may include a cooker 810, a washing machine 820, a cleaner 830, andan air conditioner 840. The mobile device 900 may include a smartphone,a remote controller, and a tap book.

The mobile device 900 includes a voice input unit 110, a firstcommunication module 918, and a position information reception unit 919.Then, the mobile device 900 further includes an input unit 916outputting information related to a function performance of the homeappliance.

The server 950 may further include a server memory 957 and a secondcommunication module 958. The service memory may store text informationmapped into an inputted voice and setting function informationcorresponding to the text information.

An application connected to the server 950 may be executed in the mobiledevice 900. Once the application is executed, a voice input mode foruser's voice input may be activated in the mobile device 900.

When a user's voice is inputted through the voice input unit 110 of themobile device 900, the inputted voice is delivered to the server 950 andthe server 950 may recognize the inputted voice to transmit a command ona setting function performance to a home appliance corresponding to avoice command. At this point, the server 950 may recognize the positionof the mobile device 900 and may then transmit a command on the settingfunction performance to the home appliance on the basis of informationon the recognized position.

Then, the server 950 may transmit the information on the settingfunction performance to the mobile device 900 and the information may beoutputted to the output unit 916 of the mobile device 900. In moredetail, information on voice recognition, position recognition, andsetting function performance may be outputted to the output unit 916 ofthe mobile device 900. That is, information described with reference toFIGS. 26, 28A and 28B may be outputted to the output unit 916.

In such a way, since a user inputs a voice command or a manipulationthrough the mobile device 900 and checks information on voicerecognition, position recognition, and setting function performance of ahome appliance through the output unit 916 of the mobile device 900,user's convenience may be improved.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, various variations and modificationsare possible in the component parts and/or arrangements of the subjectcombination arrangement within the scope of the disclosure, the drawingsand the appended claims. In addition to variations and modifications inthe component parts and/or arrangements, alternative uses will also beapparent to those skilled in the art.

The invention claimed is:
 1. A smart home appliance comprising: a voiceinput unit collecting a voice of a user, the voice input unit comprisinga microphone; a voice recognition unit recognizing a text correspondingto the voice collected through the voice input unit; a capturing unitcollecting an image for detecting a visage of the user, the capturingunit comprising a camera; a memory unit mapping the text recognized bythe voice recognition unit and a setting function and storing the mappedinformation, and storing a keyword information that the user may inputto start a voice recognition service; a control unit determining whetherto perform a voice recognition service on the basis of at least oneinformation of image information collected by the capturing unit andvoice information collected by the voice input unit; a filter unitremoving a noise sound from the voice inputted through the voice inputunit; and a memory unit mapping voice information related to anoperation of the smart home appliance and voice information unrelated toan operation of the smart home appliance in the voice inputted throughthe voice input unit and storing the mapped information; wherein thecontrol unit comprises a face detection unit recognizing that the useris in a staring state for voice input when image information on theuser's visage is collected for more than a setting time through thecapturing unit, and wherein the control unit determines that a voicerecognition service standby state is entered when it is recognized thatthere is keyword information in the user's voice collected through thevoice input unit and the user is in the staring state through the facedetection unit.
 2. The smart home appliance according to claim 1,further comprising: a region recognition unit determining a region ofthe user from among a plurality of regions of the user on the basis ofinformation on the user's voice collected through the voice input unit;and an output unit outputting region customized information on the basisof information on the region determined by the region recognition unitand information on the setting function.
 3. The smart home applianceaccording to claim 2, wherein the setting function comprises a pluralityof functions divided according to the plurality of regions; and theregion customized information including one function matching the regioninformation determined by the region recognition unit from among theplurality of functions is outputted through the output unit.
 4. Thesmart home appliance according to claim 2, wherein the output unitoutputs the region customized information by using a dialect in theregion determined by the region recognition unit.
 5. The smart homeappliance according to claim 2, wherein the output unit outputs a keyword for security setting and the voice input unit sets a reply wordcorresponding to the key word.
 6. The smart home appliance according toclaim 1, further comprising an emotion recognition unit and an outputunit, wherein the voice recognition unit recognizes a text correspondingto first voice information in the user's voice collected through thevoice input unit; the emotion recognition unit extracts an emotion ofthe user on the basis of second voice information in the user's voicecollected through the voice input unit; and the output unit outputs usercustomized information on information on the user's emotion determinedby the emotion recognition unit and information on the setting function.7. The smart home appliance according to claim 6, wherein the firstvoice information comprises a language element in the user's voicecollected through the voice input unit; and the second voice informationcomprises a non-language element related to the user's emotion.
 8. Thesmart home appliance according to claim 6, wherein the emotionrecognition unit comprises a database where information on voicecharacteristics of the user and information on an emotion state aremapped; and the information on the user's voice characteristicscomprises information on a speech spectrum having characteristics forthe user's emotion.
 9. The smart home appliance according to claim 6,wherein the setting function comprises a plurality of functions to berecommended or selected; and the user customized information includingone function matching the emotion information determined by the emotionrecognition unit from among the plurality of functions is outputtedthrough the output unit.
 10. The smart home appliance according to claim1, further comprising: a position information recognition unitrecognizing position information; and an output unit outputtinginformation on the setting function on the basis of position informationrecognized by the position information recognition unit.
 11. The smarthome appliance according to claim 10, wherein the position informationrecognition unit comprises: a GPS reception unit receiving a positioncoordinate from a position information transmission unit; and a firstcommunication module communicably connected to a second communicationmodule equipped in a server.
 12. The smart home appliance according toclaim 10, wherein the output unit comprises a voice output unitoutputting the information on the setting function as a voice, by usingthe position information or a dialect.
 13. The smart home applianceaccording to claim 10, wherein the output unit outputs informationoptimized for a region recognized by the position informationrecognition unit among a plurality of information on the settingfunction.
 14. The smart home appliance according to claim 10, whereinthe position information recognized by the position informationrecognition unit comprises weather information.
 15. An operating methodof the smart home appliance according to claim 1, the method comprising:collecting the user's voice through the voice input unit; recognizingwhether keyword information is included in the collected voice;collecting image information on the user's visage through the capturingunit; and entering the standby state of the voice recognition service onthe basis of the image information on the user's visage.
 16. Theoperating method according to claim 15, further comprising performing asecurity setting, wherein the performing of the security settingcomprises: outputting a predetermined key word; and inputting a replyword in response to the outputted key word.
 17. The method according toclaim 15, further comprising: extracting an emotion state of the user onthe basis of information on the user's voice collected through the voiceinput unit; and recommending an operation mode on the basis ofinformation on the user's emotion state.
 18. The method according toclaim 15, further comprising: recognizing an installation position ofthe smart home appliance through a position information recognitionunit; and driving the smart home appliance on the basis of informationon the installation position.
 19. The method according to claim 18,wherein the recognizing of the installation position of the smart homeappliance comprises receiving GPS coordinate information from a GPSsatellite or a communication base station.
 20. The method according toclaim 18, wherein the recognizing of the installation position of thesmart home appliance comprises checking a communication address as afirst communication module equipped in the smart home appliance isconnected to a second communication module equipped in a server.
 21. Avoice recognition system using a mobile device comprising a voice inputunit having a microphone for collecting a voice of a user and a mobiledevice communication module; and a smart home appliance comprising: avoice recognition unit recognizing a text corresponding to the voicecollected through the voice input unit; a capturing unit collecting animage for detecting a visage of the user, the capturing unit comprisinga camera; a memory unit mapping the text recognized by the voicerecognition unit and a setting function and storing the mappedinformation, and storing a keyword information that the user may inputto start a voice recognition service; a filter unit removing a noisesound from the voice inputted through the voice input unit; and a memoryunit mapping voice information related to an operation of the smart homeappliance and voice information unrelated to an operation of the smarthome appliance in the voice inputted through the voice input unit andstoring the mapped information; and a control unit comprising a facedetection unit recognizing that the user is in a staring state for voiceinput when image information on the user's visage is collected for morethan a setting time through the capturing unit; and a smart homeappliance communication module, wherein the control unit determines thata voice recognition service standby state is entered when it isrecognized that there is keyword information in the user's voicecollected through the voice input unit and the user is in the staringstate through the face detection unit, wherein the smart home applianceoperates based on the user's voice collected through the voice inputunit and the respective communication module equipped in each of themobile device and the smart home appliance, and wherein the mobiledevice further comprises a movement detection unit determining whetherto enter the standby state of the voice recognition service in the smarthome appliance by detecting a movement of the mobile device.
 22. Thevoice recognition system according to claim 21, wherein the movementdetection unit comprises an acceleration sensor or a gyro sensordetecting a change in an inclined angle of the mobile device, whereinthe voice input unit is disposed at a lower part of the mobile device.23. The voice recognition system according to claim 21, wherein themovement detection unit comprises an illumination sensor detecting anintensity of an external light collected by the mobile device.